1. Field of the Invention
The present invention relates to a technical field of red eye compensation through image processing, and more specifically to a red eye compensation method with which it is possible to perform red eye correction with efficiency, a red eye compensation method with which it is possible to perform red eye compensation at the time of remaking or reprinting or the like with efficiency, an image processing apparatus for implementing the red eye compensation method, and an image processing method with which it is possible to perform red eye compensation processing in a process of printing with film processing.
The present invention also relates to a technical field of image quality deterioration correction such as red eye correction and aberration correction through image processing. In particular, the present invention relates to an image processing method in which image quality deterioration correction such as red eye correction is performed efficiently through preprocessing; a printing method for outputting a print subjected to image processing such as red eye correction; a printer which is adapted for implementing the printing method and has a preprocessing function; and a direct printer for directly outputting image data read out of a digital camera or its recording medium as a print without loading the data into a PC (personal computer).
2. Description of the Related Art
Heretofore, images photographed on photographic films, such as negative films and reversal films, (hereinafter simply referred to as “films”) have been commonly printed on light-sensitive materials (photographic paper) through so-called direct exposure in which the film images are projected onto the light-sensitive materials to achieve exposure.
Also, in recent years, digital photoprinters have been developed and put into practical use. Each digital photoprinter outputs images recorded on a film as prints by photoelectrically capturing the images on the film, converting the captured images into digital signals, subjecting the digital signals to various kinds of image processing to generate image data for recording, and exposing a light-sensitive material using recording light modulated in accordance with the image data for recording.
The digital photoprinter basically includes: a scanner (image reading apparatus) that brings reading light into incident on a film and reads the light projecting the film, thereby photoelectrically capturing images recorded on the film; an image processing apparatus that analyzes digital image data generated by the image capturing by the scanner (image analysis) and performs image processing corresponding to a result of the analysis on the digital image data, thereby obtaining image data for outputting; a printer (image recording apparatus) that exposes a light-sensitive material by, for instance, performing light beam scan in accordance with the image data outputted from the image processing apparatus, thereby recording a latent image; and a processor (developing apparatus) that performs development processing on the light-sensitive material exposed by the printer so as to obtain (finished) prints in which the images are reproduced. Here, an input machine is formed by the scanner and the image processing apparatus, while an output machine (printer/processor) is formed by the printer and the processor.
The digital photoprinter obtains digital image data by photoelectrically capturing images photographed on a film and performs image processing and exposure of a light-sensitive material. Therefore, the digital photoprinter is capable of creating prints not only from images photographed on a film but also from images (image data) photographed with a digital camera or the like.
Also, image processing is performed through image data processing, so that it is possible not only to very favorably correct color and density but also to obtain a high-quality image by performing image processing, such as gradation correction and sharpness processing (sharpness enhancement processing), that has been basically impossible to attain with an ordinary direct exposure printer.
Further, the digital photoprinter is not limited to the outputting of image data as prints and it is also possible to record the image data on a recording medium, such as a CD-R, as image files.
By the way, the most important factor that determines the quality of an image, such as a portrait, that contains a human subject is how the human subject appears in a finished print. Therefore, a so-called red eye phenomenon, in which the eyes (pupils) of a photographed human subject are colored in red due to the influence of light emitted from an electronic flash at the time of photographing, constitutes a serious problem.
In the direct exposure photoprinter, in order to compensate red eye, it is required to retouch a film using a color material or the like, and this film retouching is performed by a skilled operator at much expense in time and effort. Therefore, it is substantially impossible to perform the red eye compensation at the time of printing with film processing or the like.
In contrast to this, with the digital photoprinter, it is possible to perform the red eye compensation through image analysis and image data processing, meaning that the above film retouching is not required at all. For instance, each pupil colored in red is detected through cutting-out and position designation by an operator, automatic extraction utilizing image analysis, and the like. Then, the detected pupil (its image data) is subjected to color transformation in order to correct the red eye. In this manners red eye compensation processing is performed.
Such red eye compensation is time-consuming processing because a relatively large amount of calculation needs to be performed. Therefore, under the present circumstances, it is technically difficult to perform full-automatic red eye compensation with reliability on the image in every frame as a predetermined processing target. Consequently, it is required to use some kind of function for performing selection, confirmation, addition, and correction.
Thus, it is usual in a print shop using a digital photoprinter provided with a red eye compensation function that a print with red eye phenomenon which may possibly be found out at the time of inspection or the like in a process of printing with film processing is regarded as an NG print and remaking of the relevant frame (fresh printing with respect to the NG print) is performed, thereby providing a customer with a print having no red eye problem. There is another case where, when a customer requests reprinting (additional printing), he/she also requests red eye compensation.
When red eye compensation is performed in a digital photoprinter at the time of such remaking or reprinting, it is required to photoelectrically re-capture the film with the aforementioned scanner and to perform completely the same processing as at the time of printing with film processing before performing the red eye compensation processing.
That is, in the digital photoprinter, when remaking or reprinting is performed in order to perform red eye compensation processing, it is required to perform completely the same operations as in the case of ordinary print outputting or the like on frames whose prints have been outputted once. This lowers operability and efficiency and therefore constitutes a factor of reduction in productivity.
Therefore, as described above, even with the digital photoprinter having the red eye compensation function, it is difficult to carry out red eye compensation processing at the time of so-called printing with film processing or the like from the viewpoint of productivity, cost, and the like. Consequently, under the present circumstances, red eye compensation processing is mainly carried out only in a remaking process or a reprinting process.
However, it may also be possible to perform full-automatic red eye compensation in which operator's manipulations are not required.
Incidentally, as a red eye compensation method used in the digital photoprinter to make correction to the red eye, there is known a method with which the face of a human subject is extracted from an image through, for instance, edge detection, hue detection, or the like, each pupil colored in red is detected from the extracted face, and the detected pupil (its image data) is subjected to color transformation or the like, thereby performing the red eye correction.
FIG. 13 shows an exemplary flowchart of the conventional full-automatic method of red eye compensation processing to be implemented in a digital photoprinter as described above,
The conventional full-automatic method of red eye compensation processing as shown in FIG. 13 starts with the number N of processed frames equal to 1. At Step 200 of the method, image data on an image as the processing target is read out of an image data source such as a digital camera, a recording medium, a scanner, and so forth.
At Step 202, the image data on the image as the processing target is analyzed to extract the face of a human subject from the image through, for instance, edge detection, hue detection, or the like and then detect the pupil that is colored in red, as described above. If no red eye phenomenon is detected, the method is skipped to Step 208.
If the red eye is detected at Step 204, the color transformation and other processing are performed on the pupil colored in red subsequently at Step 206 so as to make correction to the red eye.
In the case of the image not only subjected to red eye correction at Step 206 but involving no red eye phenomenon to be detected at Step 204, usual image processing such as color density correction and image structure processing (including sharpness processing) is performed on the image at Step 208.
Based on the output image data obtained by performing the above image processing, the image is reproduced and outputted as a print at Step 210.
Subsequently at Step 212, it is determined whether or not the number N of processed frames has attained the number of processed frames intended for one case (one processing unit), N0, which is equal to 50, for instance (N≧N0=50). If the number N of processed frames is smaller than N0 (50), it is incremented by 1 (N=N+1) at Step 214 before the method goes back to the image reading process at Step 200. Red eye compensation (detection and correction), image processing, and print outputting are repeated until the number N becomes equal to N0 (50), that is to say, terminated when N has attained N0 (50).
Generally, red eye detection requires about 1.5 seconds per frame and red eye correction about 0.3 seconds per frame (as measured on the PC with a Pentium® 4 CPU 2 GHz on board) even when the red eye compensation processing is carried out in a large scaled digital photoprinter dedicated to photographic printing. When the full-automatic method of red eye compensation processing as above is used for outputting all the images (in the frames intended for one case, the number N0 of which is equal to 50, for instance) as prints, the red eye detection process requires 50 frames×1.5 seconds=75 seconds because the red eye detection is performed on all the images (in the 50 frames). Assuming that the red eye phenomenon has been found out in 9 out of 50 frames in the red eye detection process, it takes 77.7 seconds in total to carry out red eye compensation processing in accordance with the conventional flowchart as shown in FIG. 13 because the red eye correction time of 9 frames×0.3 seconds=2.7 seconds is added to the red eye detection time of 75 seconds.
In view of the fact that the processing time is increased by up to 77.7 seconds for just 50 frame images, it is not negligible upon outputting a large number of images as prints that an additional processing time of 1.8 seconds is required for each frame image. It can not be expected to employ such a high-performance CPU as above for printers, especially those of consumer use, because there is a severe cost reduction race with respect to printers, so that a longer processing time will practically be required.
As described above, it is possible indeed to automatically perform red eye compensation processing in digital photoprinters. In that case, however, the red eye compensation processing, as comprising the red eye detection process in which face extraction and red eye detection are performed and the red eye correction process in which correction is made to the red eye, requires a large amount of calculation. In addition, it is difficult to perform the above processes using image data for condition setting and verification image creation (so-called prescanned data) which is small in data amount, and it is required to perform the processes using image data for outputting (so-called fine scanned data) which is large in data amount.
Further, the number of human subjects in one frame (one image) is not limited to one and, when multiple human subjects exist in the same frame, it is required to perform the red eye detection and the red eye correction on every human subject.
When full-automatic red eye compensation processing is performed in the digital photoprinter, the above processes are performed on all the frames, the number of which is 24, for instance. In general, however, it is a rare case that the red eye problem occurs in every frame. Therefore, in the full-automatic red eye compensation processing, the red eye detection process is unnecessarily performed on frames having no red eye problem, so that it takes a very long time to carry out the processing (meaning that a long time is wasted). In other words, the full-automatic red eye compensation processing raises a problem of reduction in processing performance, namely productivity, of the digital photoprinter, although it saves the trouble of an operator.
There may be another problem of erroneous detection where frames having no red eye problem are mistaken as having such problem.
For this reason, it is general at present that the red eye compensation processing is performed not at the time of printing with film processing but at the time of remaking or reprinting only on the relevant frames, especially those designated or selected by a customer upon a possible request for red eye compensation. In that case, the red eye compensation processing may be carried out manually in accordance with the result of visual verification by an operator or, alternatively, in a semi-automatic manner that, for instance, the frames of interest are manually designated and the red eye detection and correction processes are automatically performed on the designated frames.
It was proposed to record various kinds of photographing information concerning photographed images, which are obtained by photographing a subject with a digital camera, in a recording medium (memory card) of the camera along with the image data on the photographed images and utilize the information at the time of print outputting. Information to be recorded along with image data at the time of photographing has been standardized as Exif (standards). Particularly, contents to be recorded are defined in detail by Exif 2.2 aiming at their plenitude and commonality so that the quality of image printing can be readily improved by utilizing the information recorded at the time of photographing.
Thus, it is possible under Exif 2.2 to carry out optimal printing on the basis of various kinds of photographing information. To be more specific: In the case of a digital still camera (DSC), various kinds of photographing information set by a photographer such as “information about photographing condition” and “information about photographed scene” are recorded along with “image data” in the camera as an Exif tag. In a printer, on the other hand, such photographing information is accurately read with certain application software so as to perform optimal printing. Thus, an image in line with the intention of the photographer can be reproduced by obtaining various kinds of photographing information, which heretofore could not but be estimated, directly from the digital camera.
The photographing condition set on the DSC is accurately transmitted to the printer and processing is carried out optimally. For instance, if the mode of exposure was set to “automatic” upon photographing and the photographed image is of underexposure or overexposure due to something not intended by the user, density correction is so performed on the image that the exposure is optimized. In the case of manual mode, no correction is made in the printer because it is indicated that the exposure of interest was set by the photographer with some intention or other. Similarly, if the white balance was set to “automatic”, processing is carried out so as to remove color fog, if any. If the photographed image has a wash-out as a result of automatic light emission from an electronic flash, such processing as for the reduction in brightness of too bright areas is performed on the image so as to optimize its density.
In the printer, the information about the scene photographed with the DSC and recorded therein is utilized for printing so as to obtain a photograph suited to the photographed scene. For instance, if a photographing mode suited to the subject was selected upon photographing (from among such modes as “standard”, “landscape”, “person” and “night view”), the information about the mode can be exploited for the printing more suitable to the photographed scene. In the case of “landscape” mode being selected, the contrast or saturation is made higher, or color reproduction processing which is preferred by people is performed using stored colors such as blue and green. In the case of “night view” mode, bright areas are intensified, with dark areas being maintained as such. A night view is regenerated more vividly by making correction to color fog or contrast or intensifying the saturation.
When Exif information is utilized to optimize conditions for image processing as described above, printers are required to have an extremely high image processing ability in order to support Exif 2.2. For this reason, Exif 2.2-supporting printers are limited to printers each equipped with an image processing apparatus (image processor) having a high image processing speed or image processing ability, for instance, a PC (personal computer) or one or more CPUs (MPUs), especially to large-scaled digital photoprinters. Even in the case of such Exif 2.2-supporting printers with a high image processing speed or image processing ability, however, few of them can perform red eye compensation processing and, in addition, the red eye compensation processing is troublesome and time-consuming and consequently decreases productivity even in large-scaled digital photoprinters. In the present situation, therefore, the red eye compensation processing is performed on a special occasion, for instance, at the time of remaking or reprinting, in a manual or semi-automatic manner.
In direct printers which are commercially available, image data on photographed images are directly read out of a digital camera without temporary loading into a PC (personal computer) and prints of photographic quality are outputted as hardcopy images.
FIG. 14 shows a conventional direct printer. A direct printer 150 as shown comprises an image data acquiring section 152, an image processing section 154, a print output processing section 156, a printer controlling section 158 and a recording section 160.
In the direct printer 150 as shown in FIG. 14, image data is initially read out of a memory card inserted into a card slot 152a by a card reader (card drive) provided in the image data acquiring section 152 or, alternatively, read out of a memory card loaded onto a digital camera by connecting an external input terminal 152b with the digital camera via a cable and controlling the camera with an external input processing unit of the image data acquiring section 152.
The read image data is subjected to usual image processing such as color density correction processing and processing for correction to sharpness and other image structure in the image processing section 154.
Then, in the print output processing section 156, the data is subjected to the processing for conversion into image data to be outputted as a print. Using the converted data, the recording section 160 provided with an ink-jet recording head and so forth is controlled by the printer controlling section 158 to record a reproduced image on such paper as ink-jet recording paper, a hardcopy image as a print being thus outputted.
Conventional direct printers as above have only a limited image processing ability or speed and no such printer can perform the red eye compensation processing that is troublesome and time-consuming. Some direct printers have been commercialized indeed as an Exif 2.2-supporting printer which is required to have a high image processing ability or speed as described before and examples of them include BJ 895 PD with twin CPUs on board (Canon Inc.). At present, however, such direct printers are not equipped with any software or hardware for red eye compensation processing which is required to have a higher image processing ability or speed even though the printers have an image processing ability or speed allowing them to support Exif 2.2.
The respects as described above in connection with the red eye compensation processing are also true in the case of correction to image quality deterioration due to golden eye phenomenon and other defects with respect to the eyes of a subject, or even in the case of aberration correction, for instance, correction to image quality deterioration caused by an objective lens, such as distortion, lateral chromatic aberration, marginal luminosity deterioration, and defocus.